An Access-mode Prediction Technique Based on Cache Hit and Miss Speculation for Cache Design Achieves Minimal Energy

The successful pursuit of high performance on computer systems has produced the negative by-product of high power dissipation. Circuit-level techniques alone can no longer keep power dissipation under a reasonable level. Researchers have made efforts to reduce power dissipation at the architectural level by producing such schemes as reducing on-chip cache power consumption—a major power consumer in microprocessors. For example, experts believe that the processor power consumed by on-chip caches will increase from the 15 percent in Compaq’s Alpha 21264 to 26 percent in future Alpha 21464 processors. A set-associative cache is commonly used in modern computer systems for its ability to reduce cache conflict misses. However, a conventional set-associative cache implementation is not power-efficient. As Figure 1a shows, a conventional n-way set-associative cache probes all n blocks (both tag and data portions) in a set but, at most, will only really use one block. The percentage of wasted energy will increase as cache associativity n increases. High-associativity caches already exist in some commercial processors. For example, the Intel Pentium 4 processor exploits four-way L1 caches and an eight-way L2 cache. An effective approach to reduce power consumption in set associative caches is lowering the number of memory cells involved in an access. One method divides each data RAM into multiple sub-banks and only activates words at the required offset from all cache ways. Another alternative is to selectively disable a subset of cache ways during execution periods with modest cache activity, or to dynamically resize both the number of sets and ways in caches. The phased cache first compares all the tags with the accessing address, then probes only the desired data way. Way-prediction is another effective approach that speculatively selects a way to access before making a normal cache access. Figures 1b and 1c illustrate the access patterns for phased and way-prediction n-way setassociative caches. Compared with the conventional implementation, the phased cache only probes one data subarray instead of n data subarrays (each way comprises a tag subarray and a data subarray). However, the sequential accesses of tag and data will increase the cache access latency. The way-prediction cache first accesses the tag and data subarrays of the predicted way. If the prediction is not correct, it then probes the rest of tag and data subarrays simultaneously. An access in a phased cache consumes more energy and has longer latency than a correctly predicted access in way-prediction cache, but consumes less energy than a mispredicted access. Hence, when the prediction accuracy is high, the way-prediction cache is more energy-efficient than the phased cache. Zhichun Zhu Xiaodong Zhang